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SUMMARY AND CONCLUSIONS 



Problem 



The Naval Personnel and Training Research Laboratory (NPTRL) is 
currently investigating methods for training Navy enlisted men who 
are classified as Mental Grouj? IV on the basis of AFQT scores. One 
specific research effort involves the identification of training 
procedures for effectively augmenting a self-study course 
in basic mathematical operations. 

Background and Requirements 

An experimental program has been established at NPTRL to provide 
controlled conditions for training Mental Group IV personnel (those 
with scores on the Armed Forces Qualifications Test which fall between 
the 10th and 30th percentiles) . Experimental classes have been 
conducted in a variety of subject matter including basic mathematics. 

In an earlier study, an attempt was made to assess the effectiveness 
of supplementing the instructional material of the mathematics course 
with flash card drill. The mathematical content was presented both 
with and without flash card supplementation so that performance 
gains made under each condition could be compared. Unfortunately, 
consistently significant test gains were not achieved under either 
condition so a meaningful comparison of test gains could not 
be performed. All previous and subsequent classes achieved significant 
levels of improvement using the basic course materials. It was 
decided, therefore, to repeat the experiment with a new (and 
/ hopefully more typical) group of trainees. 



Sixteen Group IV personnel were given training in basic arithmetic 
operations. Part of the training was supplemented by practice with 
flash card materials and the effects of this additional instruction 
were determined. 

Findings and Conclusions 

In general, students demonstrated significant test gains on all 
performance indices. The additional practice with flash cards did 
not, however, increase test gains by a significant amount despite the 
fact that results indicated a potential for further improvement. It i 
is pointed out that in this study flash card instruction was applied 
to problems covering a broad range of complexity. Flash card 
instruction is typically applied to very basic types of mathematical 
operations and the results of this experiment should not be 
interpreted as questioning the usefulness of flash cards as typically 
employed. 
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THE EFFECTIVENESS OF FLASH 
CARDS IN A MATHEMATICS SELF-STUDY 
COURSE FOR GROUP IV PERSONNEL 

A. Introduction 



The Naval Personnel and Training Research Laboratory (NPTRL) 
is conducting a research program to develop and evaluate methods for 
training Navy personnel who achieve marginal scores on military 
selection tests . One of the content areas under investigation 
is basic arithmetic skills. An experimental mathematics course based 
on self-study methods has been developed and found effective for 
training Mental Group IV students (personnel whose scores on the 
Armed Forces Qualification Test fall between the 10th and 30th percentiles) . 
Following the development of this set of course materials, training 
modifications were introduced and their effects experimentally evaluated. 

Instructors had reported that Group IV trainees appear to experience 
difficulty in remembering, from day to day, procedures which they had 
supposedly mastered. It was hypothesized, in accordance with behavioral 
theories on the development of habit strength, that appropriate repetition 
of relevant problem-solving responses would promote retention, and thereby 
raise performance levels . 

Some care had to be given in planning how repetition was to be 
introduced. Simply repeating the same operations over and over might 
bore students and reduce motivation. Flash card instruction was 
selected as an appropriate training supplement since it provided 
repetition of training in a manner that was both different from 
and compatible with the basic self-study course. 

A preliminary investigation was undertaken to compare test gains 
achieved with and without flash card supplementation. * The results 
of this investigation were inconclusive; gains could not be compared 
because a significant level of improvement was not achieved under 
either condition. Since all previous and subsequent classes had 
achieved significant levels of improvement using the basic course 
materials, the performance of the class under investigation was judged 
to be atypical. Present research goals were to repeat the flash card 
experiment with a differ group of trainees in order to determine 
whether or not a class th. does make effective use of the basic course 
materials will benefit from additional practice with flash cards. 



Main, R. E., Development and evaluation of an experimental course 
in applied mathematics for Group IV personnel . San Diego: U. S. Naval 

Personnel and Training Research Laboratory, September 1969. (Research 
Report SRR 70-8) 
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B. Method 



1 - Subjects 



Subjects consisted of a class of 16 Group IV personnel who had 
just completed Navy recruit training. Aptitude scores taken from 
trainees' records are presented in Appendix A. 

2. Materials 



The basic instructional materials used for this study were 
developed by NPTRL specifically for training Navy Group IV personnel. 

The course work was similar to programmed instruction in that students 
worked individually at their own rate and were periodically required 
to make responses. Unlike typical programmed instruction, the 
content was not organized into frames, and feedback was provided only 
after an entire lesson was completed rather than after each separate 
response . 

The card decks used in this experiment were somewhat unique in 
that they were designed to cover operations of a complexity which 
would not typically be associated with flash card instruction. On 
some cards*, only a portion of a problem was covered. While students 
cannot be expected to work in their heads problems such as multiplying 
two 3-digit numbers, they can be expected to distinguish whether or not 
certain critical operations have been performed correctly. For example, 
they should be able to look at a multiplication problem which had been 
worked, and indicate whether or not rows have been properly indented. 

The card decks were designed for individual drill. Each problem 
was printed on the front of a white three-by-five card, with the answer 
on the back. Each specific type of problem appeared twice (with 
different numerical values) in order to promote repetition but, at the 
same time, discourage students from simply memorizing answers. (Examples 
of the 116 mathematics problems which were used on flash cards are 
displayed in Appendix B.) 

The complete deck of cards was divided into two sets. Set A 
consisted of 62 cards covering multiplication of whole numbers, 
operations with fractions, and computation of percentage problems. 

Set B consisted of 54 cards covering division of whole numbers, operations 
with decimals, and measurement operations (involving computations and 
conversions with linear, square, and cubic units) . 



The USAFI III Arithmetic Computation Test and the Arithmetic 
Operations Quiz (AOQ) were the two tests utilized to evaluate 
performance. The USAFI III is a test commonly used by the military 
services to establish ability levels for marginal personnel. The AOQ 
was constructed specifically for the purpose of determining performance 
levels of Group IV personnel on different aspects of the same basic 
arithmetic operations covered by the USAFI III. Some of the operations 
which were covered by test items were not covered by flash card instruction. 
In comparing performance gains made with and w^hout flash card 
supplementation, only those test items which were covered by flash card 
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problems were considered. Twenty-six test problems ( oblem Set A) were 
covered by Card Set A and 24 test problems (Problem Sec B) were covered 
by Card Set. B. 

3. Procedure 



Trainees worked with the basic self-study mathematics course 
for approximately one hour a day over a period of 14 days. The 
same sequence of lessons was given to all trainees but each worked at his 
own pace. Experimental treatments were administered so that all 
trainees worked with flash cards for about 10 minutes a day, but each 
trainee worked with only one set. Half of the trainees worked with 
cards from Set A and half worked with cards from Set B. Trainees were 
* instructed to go through the cards quickly and to decide on their 

response to each before checking the answer. They were encouraged to 
record the number of flash card problems they had answered each day 
and the number of errors they made. If a trainee made very few errors, 
it was suggested that he try to go through the cards more quickly. 

From past experience, it was correctly predicted that many trainees 
would not be able to cover all portions of the course. It was felt 
that slower students would profit more from working only with those 
cards which covered the more basic operations. Rather than give 
students an entire card set to work with from the start, each set of 
cards was divided into two subsets of progressively difficult operations. 
Initially each student was given only the first subset of cards to 
work with. As soon as he started to work a lesson which was relevant 
to the advanced flash card instruction, the second subset was added 
to his deck. 



C. Results 



A previous attempt to evaluate the effectiveness of flash card 
instruction was confounded by the inability of trainees to profit from 
the basic course materials. Therefore, in the present study, it is 
appropriate to examine overall test gains. Mean improvement scores were 
computed separately for each of the two achievement tests, the USAFI III 
and the AOQ. Each trainee’s improvement score was computed by subtracting 
, the number of posttest errors from the number of pretest errors. Mean 

improvement scores on both tests were found to be statistically 
significant (for the USAFI III, mean - 4.06, SD = 5.31, p<.01; for the 
AOQ, mean = 8.44, SD = 5.62, p<.001). It is clear that trainees did 
make effective use of the basic instructional materials. 

Having determined that overall performance gains were significant, 
one can ask'what gains in performance can be attributed to the use of 
flash cards? To determine the effects of flash card instruction two 
improvement scores were computed for each trainee. One score 
represented performance on test problems that (for that trainee} were 
covered by flash cards. The other score represented performance on 
problems not covered by flash cards. Each score was computed by 
summing gains (differences between pretest and posttest raw scores) 
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made over relevant items on the USAFI III and AOQ tests. Relevant 
items were limited to those test problems which were covered by flash 
card Sets A and B. 

In Table 1 a summary of test data is presented, and mean performance 
gains for treatments and problem sets are indicated. 



TABLE 1 

Mean Performance Gains for Treatments and Problem Sets 



Problem Set Subj ects Treatments Averaging 

Without With Groups and 




Averaging Groups and Sets 3.3 4.4 



An analysis of variance employing a cross-over design was used to test 
for main effects. 4 *' A summary of this analysis is presented in Table 2. 

From Tables 1 and 2 we may conclude that the addition of flash 
card instruction did not increase test gains by a significant amount 
and that trainees made greater gains over Problem Set A than Problem Set B. 

It might be questioned whether the gains made, using basic course 
materials, left room for flash card drill to cause improvement. Mean 
pretest and posttest error scores for those 50 problems covered by flash 
card instruction were, therefore, computed and compared. Before training, 
the mean number of errors made was 23.4. Following training, the mean 
number of errors made was 15.9. Although considerable improvement was 
made, the failure of flash card instruction to augment basic course 
materials can hardly be attributed to ceiling effects. 

D. Discussion 



The present research goal was to determine whether or not a class 
that does make effective use of basic course materials will benefit 



2 

Edwards, A. L. , Experimental design in psychological research . 
New York: Molt, Rinehart and Winston, 1960 



TABLE 2 



Analysis of Variance for Effects 
of Treatments, Problem Sets, and Subjects 



Source of Variation 


Sum of 
Squares 


df 


Mean 

Square 


F 


Level of 
Significance 


Treatments 
(With and without 
flash cards) 


10.2 


1 


10.2 


i.i 


>.05 


Problem Sets (A and B) 


72.0 


1 


72.0 


7.7 


<.05 


Subj ects 


237.9 


15 


15.9 


1.7 


>.05 


Error 


130.8 


14 


9.3 






Total 


450.9 


31 




> 





from additional practice with flash cards. In order to realize this 
goal, certain performance outcomes were required. Trainees had to 
achieve significant gains using the basic course materials, but still 
have room for additional improvement. The test performance of the 
present class was quite satisfactory in these respects. 

Supplementing the course with flash card drill did not increase 
performance gains by a significant amount. It should be noted that 
flash card instruction is typically employed for very basic types of 
mathematical problems such as learning multiplication tables. The 
exoeriment reported upon here was not designed to investigate that 
type of flash card usage. Therefore, results of this study should not be 
interpreted as raising questions about the usefulness of flash cards 
as typically employed in education and training. Indications are that 
flash cards may lose effectivenss when applied to complex material. 
Further investigations should be directed toward clarifying how flash 
card effectiveness varies with problem complexity for mathematical 
operations . 



Appendix A 



Pretraining Aptitude Test Scores* 



Navy Basic Test Battery Indices 



Trainee 

Identification 

Number 


General 

Classification 
Test (GCT) 


Arithmetic 
Test (ARI) 


Armed Forces 
Qualifications 
Test (AFQT) 


1 


40 


38 


19 


2 


39 


38 


18 


3 


33 


34 


16 


4 


44 


40 


21 


5 


47 


42 


23 


6 


30 


42 


24 


7 


33 


40 


19 


8 


34 


36 


20 


9 


35 


40 


28 


10 


40 


38 


28 


11 


37 


38 


14 


12 


33 


43 


27 


13 


22 


36 


15 


14 


25 


30 


12 


15 


39 


43 


23 


16 


37 


39 


15 


Mean 


37.5 


38.6 


20.1 



*The GCT and ARI are standard scores; the AFQT is a percentile score. 
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Examples of Flash Card Problems 
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